The present invention relates to computer graphics, and more particularly to interactive graphics systems such as home video game platforms. Still more particularly, to systems and methods for providing skinning or stitching (e.g., to support skeletal animation/inverse kinematic techniques) in a low cost graphics system.
Many of us have seen films containing remarkably realistic dinosaurs, aliens, animated toys and other fanciful creatures. Such animations are made possible by computer graphics. Using such techniques, a computer graphics artist can specify how each object should look and how it should change in appearance over time, and a computer then models the objects and displays them on a display such as your television or a computer screen. The computer takes care of performing the many tasks required to make sure that each part of the displayed image is colored and shaped just right based on the position and orientation of each object in a scene, the direction in which light seems to strike each object, the surface texture of each object, and other factors.
Because computer graphics generation is complex, computer-generated three-dimensional graphics just a few years ago were mostly limited to expensive specialized flight simulators, high-end graphics workstations and supercomputers. The public saw some of the images generated by these computer systems in movies and expensive television advertisements, but most of us couldn""t actually interact with the computers doing the graphics generation. All this has changed with the availability of relatively inexpensive 3D graphics platforms such as. for example, the Nintendo 64(copyright) and various 3D graphics cards now available for personal computers. It is now possible to interact with exciting 3D animations and simulations on relatively inexpensive computer graphics systems in your home or office.
A problem graphics system designers have confronted is how to efficiently model and render realistic looking animations in real time or close to real time. To achieve more interesting dynamic animation, a number of video and computer games have used a technique called inverse kinematics to model animated people and animals. Inverse kinematics allows a graphics artist to model animated objects in a hierarchical way so that movement of one part of the object causes another, connected part of the object to move.
For example if you raise your arm, you know your hand will move with your arm, and that your fingers will move with your hand. For example, inverse kinematics allows the animator to connect the torso, upper arm, lower arm, hand and fingers of a computer model so that moving the hand will cause the lower arm to move, moving the lower arm will cause the upper arm to move, etc. This is intuitive in the real world, but not all models behave this way in the world of animation.
The hierarchical model of an inverse kinematics model is sometimes called a skeleton, with each part of the skeleton being called a bone. The bones don""t need to accurately model real bones in terms of their shapexe2x80x94they can be rigid line segments. To create images using such a kinematic skeletal model, one usually attaches xe2x80x9cskinxe2x80x9d surfaces to each of the bones. Once the xe2x80x9cskinxe2x80x9d surfaces are attached, they can automatically follow the movement of the bones when the bones are moved. By modeling a human or animal as a skeleton of interconnected bones (i.e., the same way that real human beings and animals are constructed), it is possible to achieve realistic, natural-looking motion. Game animators have been able to achieve remarkably realistic animated motion using such techniques.
One weakness of skeletal animation is the way it handles joints between bones. Generally each bone is rigid, and its movement is defined by a transform. If the transforms cause the joint to bend, an unsightly gap can be created. For example, the elbow where the upper and lower arms of an animated character meet, or the shoulder where the character""s upper arm meets its torso ought to appear as natural as possible across a wide range of motion. Unnatural gaps at these points of connection mav destroy the illusion of realism.
The skin and flesh of real humans and animals at the intersection (joints) between bones is actually attached to and influenced by each of the various intersecting bones. For example if you xe2x80x9cmake a musclexe2x80x9d by closing your elbow, you will notice that the skin and flesh of your upper arm is influenced not only by your lower arm position/movement but also by your upper arm position/movement. If surfaces in joint regions of animated models are influenced by only a single bone, then some unsightly deformations may resultxe2x80x94degrading the realism and impact of the animation. People are relatively unforgiving when it comes to evaluating the realism of animated human models. The more realistic the animated model, the more you will notice (and perhaps be dissatisfied with) unnatural or unrealistic characteristics of the model""s appearance.
This weakness can be overcome using a technique called skinning, which adjusts and blends the positions of the vertices around the joint to create a continuous, flexible skin covering surface that provides a smooth transition between xe2x80x9cbonesxe2x80x9d where the bones meet one another. This transitional xe2x80x9cskinxe2x80x9d surface can adapt to the different relative positions of two or more intersecting xe2x80x9cbonesxe2x80x9d across a range of positions and relative motions. The resulting effect can significantly add to the illusion of realism.
On a more detailed level, the skin is typically defined by a mesh of vertices. Skinning is generally accomplished by allowing the position of each vertex in the mesh to be influenced by the movement of more than one bone in the skeletal animation model. The influence of different bones can be determined by assigning them different weights in computing the skin vertex position. A model can be animated by defining the movement of its skeleton, and the movements of the vertices that define the skin can be generated automatically (e.g., mathematically) by the graphics system.
Mathematical functions called matrix transformations are usually used to compute the location of each vertex in each frame of a skeletal animation. A separate transformation matrix is typically used for each bone that influences a given vertex. For example, if a skin vertex is located near the intersection of two bones, two transformation matrices are usually requiredxe2x80x94one for each of the two bones that influence that vertex. In order to make joints that flex naturally, it is desirable to allow the weightings of each matrix to vary for each vertex. Different vertex weightings for each vertex around a joint allow the vertex skin mesh to blend gradually from one bone to another (xe2x80x9cvertex skinningxe2x80x9d).
Such vertex skinning techniques for modeling animated objects have been quite successful in providing a high degree of realism. Many high end animation rendering engines and modeling tools support such techniques. However, one problem with skinning is that the matrix transformations required for vertex skinning are very computationally intensive. To provide surface information for skinning/stitching, you normally need multiple interpolation points between two vertex locations. This implies the need for additional unique transformations per texture coordinate based off a single vertex specificationxe2x80x94and unique texture transformations applied to the geometry or normal per texture. Thus, a plurality of matrix multiplications are required to accurately transform the skinning surface. The complexity increases with each additional matrix used.
Matrix multiplications are of course commonplace in graphics rendering systems. For example, it is common to provide matrix multiplication to transform model parameters from one space to another (e.g from modeling space to eye space) in order to project a 3D object representation onto a 2D viewing plane. However, real time systems typically minimize the number of matrix multiplications they perform. This is because each matrix multiplication can take many processor cycles if computed in software, and matrix multiplication hardware can require large amounts of xe2x80x9creal estatexe2x80x9d on a graphics chip. For this reason, there has not be much skinning support available in low cost real time rendering systems such as home video game platforms and inexpensive personal computer graphics accelerator cards. While the general purpose processor of such systems can be used to perform the skinning matrix multiplication calculations, such calculations are usually so time-consuming that the animator must sacrifice image complexity or speed performance if he or she wants skinning effects.
The present invention provides a solution to this problem In accordance with one aspect of this invention, a low cost computer graphics system includes hardware support for a more limited version of skinning called xe2x80x9cstitching.xe2x80x9d Stitching allows a surface to be transformed based on two per-vertex values corresponding to different vertices. An additional interpolation/matrix multiplication provides a limited (piecewise linear) version of skinning (xe2x80x9cstitchingxe2x80x9d) that does not slow down the rendering pipeline.
In accordance with an aspect of this invention, a real time graphics rendering system includes a pair of matrix multiplication dot product computational elements. The first dot product computational element in the cascade can be used in a conventional manner to perform a modelview (or other) transformation. The second dot product computation element can be used to perform an additional interpolation to provide stitching. The two dot product computation elements can operate successively on the same texture coordinate values to provide transformed xe2x80x9cstitchedxe2x80x9d texture coordinates for use in texture mapping a skin surface onto a vertex mesh.
In accordance with another aspect provided by this invention, a normalization block is provided between the two dot product computation units. The normalization function helps to avoid distortions that could otherwise occur under certain circumstances, e.g., when surfaces are deformed.
The additional dot product computation unit and associated matrix memory and the normalization block are relatively compact and do not occupy much chip real estate. They are not capable of full skinning, but a piecewise linear approximation of skinning can nevertheless be used to provide realistic interconnections between xe2x80x9cbonesxe2x80x9d of the skeletal animation model while relieving the game animation processor from performing skinning computations. The additional matrix multiplication computation unit does not adversely impact on rendering pipeline speed performance. Pipeline latency increases can be absorbed through buffering. The system allows per-vertex specification of transform matrix indices to permit selection of different transformation matrices on a vertex-by-vertex basis.
The second matrix multiplication computation unit provided in the example implementation can be used for applications other than stitching. For example, various types of environment and/or reflection mapping can be performed using this additional texture coordinate matrix transformation.